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SUMMARY 


Self-directed  learning  is  that  type  of  learning  which  is  not 
structured  for  the  student  by  an  instructor.  Instead,  the  student 
must  structure  his  learning  himself  by  making  decisions  about  which 
materials  are  relevant  to  his  learning  goals,  which  materials  require 
the  prior  understanding  of  which  other  materials,  and  so  on.  A 
computer-based  system  has  been  developed  to  train  students  in  this 
type  of  learning. 

A revised  system  based  on  an  earlier  version  of  a computer-based 
self-directed  learning  system  was  developed.  The  improved  system 
described  herein  contains  features  designed  to  make  it  easier  for  students 
to  use.  In  addition,  pedagogical  features  of  the  training  system  have 
been  improved,  to  give  students  an  opportunity  to  learn  the  system 
completely. 

An  experimental  test  of  the  improved  system  was  designed  to  separ- 
ate out  the  effects  of  training  in  self-directed  learning  from  the  use 
of  the  system  itself.  Data  were  collected  on  four  different  measures 
of  learning:  effective  learning,  selective  learning,  planning,  and 
verbal  report.  Results  of  the  experiment  found  that  there  were  no 
significant  differences  among  treatment  groups  in  the  performance  data 
(the  first  three  learning  measures),  even  though  one  of  the  experimental 
groups  outperformed  the  other  groups  in  every  measure.  On  the  measure 
of  verbal  report,  however,  this  experimental  group  performed  signifi- 
cantly better  than  did  the  control  group. 
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I.  INTRODUCTION 

Self-directed  learning  is  that  learning  which  the  student  directs 
himself;  that  is,  no  structure  is  imposed  on  his  learning  externally. 

This  type  of  learning  is  found  in  many  on-the-job  learning  contexts  in 
which  a worker  must  make  his  own  decisions  about  just  what  information 
will  help  him  solve  the  problem,  but  he  must  not  waste  his  time  reading 
irrelevant  information.  He  must  also  structure  his  learning  in  such  a 
way  that  if  he  encounters  unfamiliar  concepts,  he  can  locate  the  prerequi- 
site information  he  needs  to  understand  those  concepts.  Self-directed 
learning  thus  differs  dramatically  from  the  type  of  learning  that  goes 
on  in  the  classroom,  in  which  an  instructor  is  available  to  direct  the 
student's  learning.  Obviously,  skills  and  strategies  necessary  for  self- 
directed  learning  are  seldom  if  ever  taught  in  the  classroom. 

We  have  developed  a computer-based  self-directed  learning  system, 
described  in  Munro,  Rigney,  and  Crook  (1978)  and  in  Rigney,  Munro,  and 
Crook  (in  press).  Those  reports  described  an  early  version  of  this 
system  and  the  results  of  a pilot  experiment  using  the  system.  This 
report  will  discuss  a revised  version  of  the  system  and  the  results  of  a 
larger  experiment. 

In  the  pilot  experiment,  some  of  the  subjects  used  the  self-directed 
learning  aid  in  solving  a complex  problem,  while  the  other  (control)  subjects 
used  a simplified  version  of  the  learning  aid  which  did  not  include  any 
of  the  self-monitoring  features  of  the  complete  Aids  system  (described 
below).  (In  effect,  the  control  system  is  nothing  more  than  a computer- 
based  book;  as  soon  as  the  student  selects  a title  in  the  table  of  contents, 
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he  is  shown  the  corresponding  information. ) Data  were  kept  on  subjects 
in  both  the  experimental  and  control  groups  while  they  solved  a complex 
learning  task.  Various  measures  were  taken,  such  as  time  required  to  solve 
the  task,  the  number  of  errors  made,  and  several  measures  of  selectivity 
and  planning.  Results  showed  that  while  there  were  no  significant  dif- 
ferences between  the  groups,  the  experimental  group  on  the  whole  performed 
slightly  better  than  the  control  group.  More  importantly,  however, 
experimental  subjects  at  the  low  end  of  an  ability  scale  (as  measured  by 
the  Nelson-Denny  test  of  reading  ability)  performed  better  than  control 
subjects  in  the  same  range.  It  was  therefore  concluded  that  the  self- 
directed  learning  Aid  helps  students  who  are  poor  readers  to  a greater 
extent  than  it  helps  students  who  are  good  readers.  The  results  of  this 
experiment  and  verbal  and  written  comments  made  by  the  subjects  suggested 
certain  improvements  that  could  be  made  both  in  the  automated  learning  Aid 
and  in  the  training  program  that  accompanied  it.  These  improvements  are 
incorporated  into  the  current  version  of  the  Aids  system. 

The  self-directed  learning  Aids  system  consists  of  the  following 
"pages"  or  areas  of  the  computer  program: 

TASK  page:  contains  a statement  of  the  student's  learning 
task 

OBJECTIVES  page:  allows  the  student  to  break  down  the  task 
into  a set  of  sub-goals,  called  objectives 
CONTENTS  page:  contains  a list  of  the  titles  of  information 
sources  available  to  the  student 
RELEVANT  CONTENTS  page:  lists  only  those  titles  that 
the  student  has  selected  as  relevant  to  the  task. 
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From  any  one  of  these  pages,  the  student  can  go  to  any  of  the  others  as 
he  wishes;  rather  than  following  a strict  linear  order  through  the 
program,  the  student  is  free  to  move  around  as  he  deems  necessary.  Each 
page  has  a number  of  functions  which  the  student  may  use  while  on  that 
page.  For  example,  the  RELEVANT  CONTENTS  page  lists  all  those  titles  of 
information  sources  that  the  student  has  selected  to  read.  From  this  page 
the  student  may  elect  to  read  one  of  these  information  sources,  check  it 
off  if  he  feels  it  really  is  relevant  to  his  learning  task,  X it  off  if 
he  feels  it  is  irrelevant,  or  express  dependencies  between  two  information 
sources  (that  is,  state  that  one  source  should  be  read  before  the  other). 

The  current  version  of  the  automated  Aids  system  differs  from 
that  described  in  Munro,  Rignev,  and  Crook  (1978)  in  a number  of  ways. 

New  functions  have  been  added  to  some  of  the  pages,  the  goal  stack  has 
been  extensively  revised,  and  the  matching  section  has  been  eliminated 
altogether.  Previously,  students  were  allowed  to  proceed  in  either  a 
conceptually  driven  (top-down)  or  data-driven  (bottom-up)  fashion, 
although  the  system  more  easily  provided  for  the  latter.  That  is,  students 
would  select  a title  and  then  be  asked  in  the  matching  section  to  decide 
which  of  the  objectives  the  title  was  relevant  for.  There  was  no  overt 
mechanism  available  to  them  to  select  an  objective  first  and  search  for 
titles  to  help  them  attain  that  objective  (although  they  could  of  course 
do  this  mentally).  Many  students  felt  that  this  matching  procedure  was 
"backwards,"  so  the  current  version  of  the  automated  Aid  was  adopted. 

A student  is  now  led  to  proceed  in  a conceptually  driven  fashion  by  first 
selecting  an  objective  to  work  on,  then  selecting  one  or  more  titles  for 
that  objective.  Since  the  match  is  made  as  soon  as  the  student  selects 
a title,  there  is  no  need  for  a separate  matching  section  in  the  Aids 
system. 
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Another  feature  of  the  Aids  system,  the  goal  stack,  was  modified 
to  make  it  easier  for  students  to  use.  Since  there  is  limited  space  on 
the  screen  of  the  computer  terminal,  only  the  number  of  goals  (objectives 


and  titles  of  information  sources)  are  displayed  in  the  goal  stack.  A 
feature  was  added  to  the  current  version  to  allow  the  student  to  touch 
any  of  these  numbers  on  the  screen  and  be  shown  that  goal  written  out  in 
full.  The  student  may  also  elect  to  see  all  the  other  goals  which  are 
related  to  that  goal  written  out  in  full,  along  with  the  dependency  rela- 
tionships they  form  with  the  original  goal.  New  functions  which  were 
added  include  a "Task  Analysis"  box  on  the  TASK  page  which  helps  the 
student  formulate  objectives  for  the  specific  task  domain  he  is  working 
in,  and  a "Select"  box  on  the  OBJECTIVES  page  which  allows  the  student  to 
select  an  objective  to  work  on. 

In  addition  to  improvements  made  to  the  self-directed  learning  Aid 
itself,  several  improvements  were  made  to  the  training  program  which 
accompanies  it.  (The  Aids  system  is  a rather  complex  system  which  requires 
a number  of  hours  of  training  in  its  use  before  students  feel  comfortable 
with  it.)  Long  expository  passages  in  the  original  version  have  been 
broken  down  into  shorter  screen  displays,  and  short  quizzes  appear  much 
more  frequently  than  they  did  in  the  earlier  training  program.  In  addition, 
each  training  session  now  begins  with  a quiz  reviewing  the  contents  of 
the  previous  session.  These  changes  were  designed  to  keep  the  student  more 
actively  involved  with  the  training  program. 

It  was  noted  after  the  pilot  experiment  that  experimental  subjects 
had  a more  difficult  learning  task  than  control  subjects  did.  Each  training 
session  included  sample  problems  to  practice  using  the  automated  Aids  system, 
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but  only  the  experimental  subjects  had  to  learn  to  use  the  Aids  systems 
in  addition--the  control  version  of  the  "Aids"  system  required  no  training 
beyond  the  initial  session.  Consequently,  the  current  training  program 
used  simple  practice  problems,  allowing  the  students  to  concentrate  fully 
on  the  details  of  the  Aids  system.  These  simple  problems  involved  deci- 
sions concerning  maritime  rules  of  the  road.  The  final  (testing  phase) 
session  of  the  training  program,  however,  used  a complex  learning  task. 
This  task  required  students  to  troubleshoot  or  debug  a simulation  of  a 
sentence  generator  having  one  "faulty"  component. 

Finally,  the  effects  of  training  in  self-direction  and  the  use  of 
the  automated  Aid  were  separated  in  this  study.  The  pilot  experiment 
compared  experimental  subjects  who  had  both  training  and  the  Aids  system 
against  control  subjects  who  had  neither.  The  present  experiment  compared 
the  results  of  three  different  groups  to  evaluate  the  effects  of  both 
training  and  the  use  of  the  automated  Aids  system. 


II.  EXPERIMENT 


> 


An  experiment  was  conducted  to  test  the  effects  of  the  self- 
directed  learning  Aid  discussed  in  Section  I. 


Experimental  Design 

The  experiment  was  designed  to  test  the  following  two  research 
hypotheses:  (1)  students  who  are  trained  in  self-direction  strategies 
will  solve  a complex  learning  task  more  quickly  and  efficiently  than 
students  who  are  not  trained  and  (2)  when  students  are  trained  in  self- 
direction  strategies,  those  students  who  are  provided  with  the  automated 
Aids  system  to  facilitate  the  use  of  those  strategies  will  solve  the  task 
more  quickly  and  efficiently  than  the  students  who  are  not  provided  with 
the  Aids  system.  Subjects  were  assigned  at  random  to  one  of  the  following 
condi tions : 

(I)  Training  plus  Aids  system:  Students  were  trained  in  self- 
direction  strategies  and  in  the  use  of  the  Aids  system.  During  the  final 
session  (during  which  data  on  student  performance  was  collected),  they 
were  provided  with  the  Aids  system  designed  to  facilitate  application 

of  the  strategies. 

(II)  Training  and  no  Aids  system:  Students  were  trained  in  self- 
direction  strategies  and  in  the  use  of  the  Aids  s;  »tem  in  the  same  way 

as  Group  I.  During  the  final  session,  they  were  not  given  the  Aids 
system;  instead,  they  had  to  accomplish  the  task  using  the  same  apparatus 
available  to  the  group  below. 

(III)  No  training  and  no  Aids  system:  Students  were  given  no 
training  in  self-direction  strategies,  nor  did  they  have  the  Aids  system 
available  during  the  final  problem-solving  session. 
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The  first  research  hypothesis  is  tested  by  comparing  the  performance 
of  students  in  Condition  II  with  that  of  students  in  Condition  III,  while 
the  second  hypothesis  is  tested  by  comparing  the  performance  of  students 
in  Condition  I with  that  of  students  in  Condition  II.  This  design  is 
similar  to  a 2 x 2 factorial  except  that  the  cell  corresponding  to  "No 
training  and  Aids  system"  was  omitted  because  students  could  not  be 
expected  to  use  the  complex  Aids  system  without  prior  training  in  its  use. 

During  the  testing  phase,  students  in  the  training-only  and  control 
groups  (Conditions  II  and  III)  use  an  automated  system  containing  only 
the  Task  and  Contents  pages  of  the  system  described  in  Section  I.  Students 
in  these  groups  have  the  same  learning  task  and  the  same  information  to 
read,  but  they  have  none  of  the  Aids  system  available  to  students  in  the 
training-plus-Aids  group  (Condition  I).  (Information  sources  in  this 
simplified  system  are  accessed  directly  from  the  Table  of  Contents.  As 
soon  as  the  student  touches  a title,  he  is  shown  the  corresponding 
information  source. ) 

The  training  sequence  for  control  subjects  is  similar  to  that  for 
the  other  students,  except  that  the  basic  system  is  never  modified  for 
them,  so  there  is  no  need  for  teaching  sessions  other  than  the  initial 
one.  Thus,  sessions  two  and  three  are  practice  sessions  using  the 
control  system,  and  session  four  is  the  testing  phase.  The  training 
phase  for  students  in  the  training-only  group  is  identical  to  that  for 
students  in  the  training-plus-Aids  group.  During  the  final  session, 
however,  the  training-only  students  use  the  control  system  rather  than 
the  self-directed  learning  Aids  system.  The  combinations  of  different 
systems  used  in  all  three  conditions  are  shown  in  Figure  1. 
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Condition 


Training 


Test 


I II  III 


Ai  ds 

’ ‘ 1 

Aids 

Control 

System 

System 

System 

Aids 

Control 

Control 

System 

System 

System 

Figure  1.  Combinations  of  Systems  Available  to 


Subjects  in  Each  Condition. 


* 


Method 


Subjects  for  the  experiment  were  volunteers  recruited  from  a 
lower-level  NROTC  course  at  a major  university.  Thirty-nine  subjects 
were  assigned  at  random  to  each  of  the  three  conditions  discussed  above. 
The  students  were  told  that  they  would  receive  exposure  to  junior-level 
NROTC  course  material  by  participating  in  the  experiment  as  well  as 
twenty  dollars  for  completing  the  experiment.  During  the  test  session, 
they  were  told  that  they  could  receive  an  additional  two  dollars  for 
solving  the  task  correctly  on  the  first  attempt  or  an  additional  one 
dollar  for  solving  it  on  the  second  attempt.  The  reason  for  this  bonus 
was  to  discourage  random  guessing  and  to  encourage  students  to  have  a 
fair  degree  of  confidence  in  their  answer. 

Data  Collection 

The  data  collected  during  the  final  session  were  designed  to 
measure  both  effective  learning  and  self-directed  learning.  Effective 
learning  is  defined  in  terms  of  the  time  spent  reading  information  and 
the  number  of  errors  made  in  attempting  to  solve  the  task.  Data  are 
collected  for  each  student  on  the  number  of  erroneous  attempts  (wrong 
answers)  made  when  the  student  attempts  to  perform  the  task  and  the 
total  time  the  student  spends  reading  information  sources  to  solve  the 
* task.  Self-directed  learning  is  much  more  difficult  to  measure,  but 

it  seems  to  be  typified  by  two  phenomena:  planning  and  selectivity  in 
the  use  of  available  information  sources.  The  data  collected  reflect 
operational  definitions  of  these  phenomena. 
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It  is  not  an  easy  matter  to  discover  whether  a student  is  engaged 
in  effective  planning.  One  type  of  data  saved  by  our  PLATO  program  is 
the  sequence  in  which  students  accessed  the  information  resources 
available  to  them.  Our  analysis  of  the  troubleshooting  task  presented 
to  the  students  in  the  testing  phase  has  resulted  in  the  formulation  of 
a set  of  rules  for  scoring  deviations  from  the  order  in  which  the 
information  sources  should  be  accessed.  These  rules,  which  we  call 
anti-precedence  rules,  take  the  form  of  prohibitions  of  certain  sequences. 
The  extent  to  which  a student  has  departed  from  sequences  permitted  by 
an  ideal  task  analysis  can  be  expressed  in  terms  of  the  number  of  times 
the  student's  study  sequence  violates  the  anti-precedence  rules. 

Basically,  the  anti -precedence  rules  require  that  the  student  not 
read  about  a subcomponent  of  the  sentence  generator  unless  he  has  first 
read  about  the  component  which  contains  it.  A particular  rule  is 
violated  if  the  student  either  fails  to  read  about  the  larger  component 
or  reads  about  it  after  reading  about  the  subcomponent.  These  rules  thus 
reflect  the  idea  that  a student  should  not  attempt  an  action  until  he  has 
completed  its  prerequisite. 

Selectivity  in  the  Use  of  Information  Sources 

Selectivity  has  to  do  with  reading  information  sources  which  are 
relevant  to  the  learning  task  and  not  reading  the  irrelevant  information 
sources.  The  ratio  of  relevant  information  sources  read  to  all  informa- 
tion sources  read  is  a measure  of  selectivity;  a student  for  whom  this 
ratio  is  high  has  read  primarily  relevant  sources  and  few  irrelevant 
ones.  Two  different  ratios  of  this  type  were  computed,  the  first  a ratio 
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* of  the  number  of  information  sources  read  and  the  second  a ratio  of  the 
time  spent  reading  those  information  sources.  That  is,  the  first  is  the 
ratio  of  the  number  of  relevant  information  sources  read  to  the  total 

• number  of  information  sources  read,  and  the  second  is  the  ratio  of  the 
time  spent  reading  relevant  information  sources  to  the  time  spent  reading 
all  information  sources. 

Subject  Strategy  Summaries 

In  addition  to  collecting  data  on  effective  learning  and  self- 
directed  learning,  we  attempted  to  validate  the  schema  representations 
given  in  the  Appendix.  Following  the  final  session,  students  were  asked 
to  describe  the  "learning  techniques  or  strategies"  that  they  used  in 
solving  the  complex  troubleshooting  task.  Their  summaries  of  these 
strategies  were  analyzed  using  a modified  form  of  the  method  for  scoring 
text  recalls  and  summaries  described  in  Gordon,  Munro,  Rigney,  & Lutz 
(1978),  to  try  to  determine  if  subjects  in  different  treatment  groups 
would  summarize  their  strategies  differently.  This  method  of  text 
analysis  examines  summaries  for  occurrences  of  statements  of  particular 
relevance  for  self-directed  learning,  that  is,  the  representations  pre- 
sented in  the  Appendix.  These  were  translated  into  short  English 
statements,  and  three  judges  scored  the  summaries  of  strategies  for  the 
presence  of  these  statements. 

K 

Resul ts 

For  each  of  the  three  treatment  groups,  means  and  standard  deviations 
were  computed  for  the  dependent  measures  and  individual  differences  vari- 
ables. The  means  for  the  dependent  measures  are  presented  in  Table  1,  and 
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the  means  for  the  individual  difference  measures  are  given  in  Table  2. 

A 

Visual  inspection  of  the  results  presented  in  Table  1 reveals  that  the 
training- only  group  performed  worse  than  both  the  training-plus-aids 
and  the  control  groups  on  almost  every  measure.  In  retrospect,  this 
result  seems  unsurprising,  since  the  training  in  self-direction  that 
the  subjects  were  given  was,  for  the  most  part,  either  oriented  toward 
or  interpretable  in  terms  of  the  functions  of  the  Aids  system.  Students 
in  the  training-only  group  were  required  to  solve  a complex  problem  in 
the  final  session  using  an  "aids"  system  with  which  they  were  unfamiliar. 

The  comparison  of  the  training-plus-aids  group  to  the  control  group  will 
therefore  be  emphasized  below. 

The  number  of  errors  made  in  solving  the  complex  problem  is  a 
measure  of  the  effectiveness  of  the  learning  strategies  that  students 
use  to  accomplish  their  task.  Although  a paired  comparisons  test  did 
not  reveal  a significant  difference  between  the  two  groups  of  interest, 
inspection  of  the  descriptive  statistics  indicates  that  the  control  group 
made  slightly  more  errors  on  the  average  and  varied  more  widely  in  their 
performance.  The  variance  of  the  control  group  is  about  twice  that  of 
the  treatment  group.  This  suggests  that  the  treatment  served  to  reduce 
individual  variation  in  complex  problem  solving.  A second  measure  of 
learning  effectiveness  is  the  amount  of  time  spent  reading  the  information 
resources.  The  training-plus-aids  group  spent  less  time  reading  the 
information  than  the  control  group,  but  again  the  difference  did  not  reach 
significance. 

The  other  three  variables  included  in  Table  1 are  measures  of  self- 
directedness  during  problem  solving  that  require  the  use  of  text.  All 
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Table  1 


Means  and  Standard  Devistions 
of  Dependent  Measures 


Variables 


Treatment  Conditions 


Reading  Time 
(minutes) 


Errors 


Selectivity  in 
Titles  Picked 


Selectivity  in 
Time 

A1 location 


Planning 

Violations 


Training  + Aids 

Training-Only 

Control 

X 

20.05 

41  .89 

27.44 

SD 

(14.71) 

(33.93) 

(13.12) 

X 

1 .23 

3.31 

1.62 

SD 

(1.54) 

(3.20) 

(3.02) 

X 

.77 

.61 

.68 

SD 

( .17) 

( -17) 

( .22) 

X 

.71 

.62 

.67 

SD 

( .20) 

( .18) 

( .24) 

X 

1.39 

1.54 

1 .58 

SD 

(1.50) 

(1.13) 

(1.56) 
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differences  between  the  two  groups  are  in  the  predicted  direction  but  are 
not  significant.  Students  in  the  training-plus-aids  group  were  somewhat 
more  selective  in  choosing  only  relevant  titles  and  also  spent  more  of 
their  time  reading  relevant  information  than  did  the  controls.  The 
treatment  conditions  also  resulted  in  fewer  planning  violations  than  did 
the  control  condition. 

Therefore,  as  assessed  both  by  final  performance  and  by  behaviors 
during  problem  solving,  the  treatment  condition  produced  slightly  (but 
not  significantly)  more  effective  and  self-directed  learning. 

Individual  differences  in  learner  ability  were  compared  across 
groups.  No  differences  were  found,  indicating  that  the  randomization 
procedure  successfully  yielded  similar  groups.  Scores  on  both  verbal  and 
mathematics  sections  of  the  Scholastic  Aptitude  Text  (SAT)  were  obtained, 
and  group  means  of  these  scores  are  shown  in  Table  2. 

All  variables  were  correlated  with  one  another,  and  the  overall 
correlation  matrix  is  shown  in  Table  3.  Several  relationships  are 
noteworthy.  There  were  strong  relationships  between  error  scores  and 
the  two  measures  of  selectivity,  r = -.57  and  -.51  (£<.05)  for  reading 
choice  and  time  allocation  respectively.  These  strong  negative  correla- 
tions between  errors  and  selectivity  indicate  that  greater  selectivity 
during  problem  solving  corresponds  to  fewer  errors.  This  finding  is 
important  because  it  suggests  that  selectivity,  as  operationally  defined 
in  this  study,  does  in  fact  result  in  more  effective  learning.  In  other 
words,  selectivity  does  result  in  better  problem-solving  performance. 

Another  interesting  relationship  is  that  found  between  errors 
and  reading  time,  r = .32  (£^.05).  Apparently,  as  students  spent  more 
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Table  2 


Means  and  Standard  Deviations  of 
Individual  Difference  Measures 


Variables  Treatment  Conditions 


Training  + Aids  Traininq-Onl y 


SAT: 

Verbal 

X 

588 

542 

SD 

( 85) 

( 57) 

SAT: 

Math 

X 

637 

635 

SD 

( 58) 

( 41) 

SAT. 

Compos i te 

X 

1225 

1184 

SD 

( 85) 

( 75) 
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Control 


566 
( 76) 


615 
( 64) 


1185 

(105) 


Combined  Correlation  Matrix 


time  reading  the  information,  they  tended  to  perform  worse  (make  more 
errors).  This  positive  relationship  between  reading  times  and  errors 
is  opposite  of  the  expected  relationship  between  time  and  performance-- 
namely,  that  the  more  time  a student  spends  learning  the  better  he  or 
she  should  do  on  a criterion  test.  The  relationship  observed  in  this 
study  suggests  that  studeits  who  were  more  efficient  (who  spent  less  time 
reading)  were  also  more  effective  in  that  they  solved  the  task  with  fewer 
errors.  Further  support  for  this  conclusion  is  found  in  the  inverse 
relationship  between  reading  time  and  selectivity,  r = -.28  (title  choice) 
and  £ = -.24  (time  allocation).  While  neither  of  these  correlations  is 
significant,  the  negative  tendency  suggests  that  students  who  were  more 
selective  tended  to  take  less  reading  time  to  solve  the  problem. 

Finally,  it  is  interesting  to  note  the  strong  correlations  among 
the  three  measures  of  self-direction--selectivity  in  title  choice, 
selectivity  in  time  allocation,  and  planning  violations.  The  two 
selectivity  ratios  are  strongly  related,  r = .90  (p  < .05).  This  rela- 
tionship suggests  that  students  who  selected  the  appropriate  information 
to  read  also  allocated  their  time  to  reading  that  information.  The 
selectivity  measures  are  negatively  related  to  planning  errors,  r = -.55 
and  -.64  (P  < .05)  for  title  choice  and  time  allocation,  respectively. 

The  negative  relationships  suggest  that  students  who  choose  selectively 
and  spent  most  of  their  time  reading  appropriate  information  did  so 
according  to  the  proper  sequence.  The  three  measures  of  self-direction 
used  in  this  study  were  therefore,  consistent  in  their  assessment  of 
student  behaviors. 
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titude  by  Treatment  Interactions  (ATI) 

The  combined  correlation  matrix  cannot  reveal  any  differences 
between  the  treatment  conditions.  Therefore,  correlations  between 
variables  were  examined  for  each  treatment  grouD  individually.  These 
correlations  appear  in  Tables  4,  5,  and  6.  Visual  examination  of  the 
data  indicates  an  overall  pattern  of  differences  between  the  three  treat- 
ment groups.  In  general,  the  expected  relationships  between  ability,  time, 
and  performance  occur  only  in  the  control  condition.  For  example,  we 
would  expect  students  of  higher  verbal  ability  to  need  less  reading  time 
to  solve  the  problem  (and  lower  ability  students  to  require  more  reading 
time).  Yet,  the  expected  negative  correlation  between  these  two  variables 
(verbal  ability  and  reading  time)  occurred  only  in  the  control  condition, 
r = .50,  .05,  and  -.48  for  training  + aids,  training,  and  control  groups, 
respectively.  The  correlations  between  verbal  ability  and  error  scores 
also  fail  to  be  strongly  negative  except  for  the  control  group,  r = -.18, 

-.12  and  -.71  respecti vely.  In  other  words,  only  in  the  control  condition 
was  the  negative  correlation  significant  (£  -^.05).  Ability,  then,  is 
predictably  related  to  the  dependent  measures  only  in  the  control  condition. 

The  relationship  between  ability  and  self-direction  also  seems  to 

% 

fit  our  expectations  only  in  the  control  group.  We  would  expect  students 
of  higher  verbal  ability  to  be  more  selective  and  make  fewer  planning 
violations.  Although  we  would  predict  that  verbal  ability  and  selectivity 

I 

are  positively  related,  a positive  correlation  between  ability  and  select- 
ivity in  time  allocation  occurred  only  in  the  control  condition,  r = -.38, 

-.47,  and  .55  respectively.  Similarly,  the  correlation  between  ability 
and  selectivity  in  title  choice  was  positive  only  for  control  subjects, 
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r = -.12,  -.49,  and. 46  respectively.  Furthermore,  we  would  expect  that 
verbal  ability  is  inversely  related  to  violations  in  planning,  yet  the 
obtained  correlations  did  not  match  that  prediction  except  in  the  control 
condition,  r = .27,  .44,  and  -.50  respectively.  Hence,  the  expected 
relationships  between  verbal  ability  and  measures  of  sel f-di rectedness 
did  not  hold  except  for  students  in  the  control  condition. 

Several  other  expected  relationships  held  only  for  the  control 
condition.  We  would  expect  that  selectivity  would  decrease  reading  time 
so  that  these  variables  should  be  negatively  correlated.  While  the 
correlations  are  negative  for  all  three  groups  (r  = -.21  and  -.14  for 
training-plus-aids,  r = -.08  and  -.09  for  training-only,  and  r = -.67  and 
-.67  for  control),  they  are  significant  only  in  the  control  group.  The 
final  prediction  is  that  planning  violations  should  be  related  to  errors 
because  inefficient  learners  are  probably  ineffective  as  well.  The 
correlations  are  r = .03,  -.23,  and  .59,  respectively.  In  other  words, 
the  naturally  expected  correlation  between  planning  violations  and  errors 
is  found  only  in  the  control  condition. 

The  ATI  evidence  shows  that  expected  relationships  between  ability 
and  performance  held  only  for  the  control  group.  It  seems  likely  that 
students  of  high  ability  in  the  experimental  groups  may  have  been  hampered 
* by  the  cumbersome  mechanics  of  the  Aids  system,  which  promoted  less 

efficient  strategies  than  they  would  have  used  on  their  own.  Less  capable 
students  in  the  experimental  groups,  who  may  have  had  no  useful  strategies, 
were  probably  helped  by  the  features  of  the  Aids  system.  This  explana- 
tion is  supported  by  the  reduced  variance  in  errors  found  in  the  training 
plus  aids  group  as  compared  with  the  control  group. 
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Strategy  Summary  Results 

Means  and  standard  deviations  for  the  scored  strategy  summaries 
are  given  in  Table  7.  A 2-factor  analysis  of  variance  (ANOVA)  with 
repeated  measures  on  the  rater  factor  was  performed  on  these  data,  and 
the  results  appear  in  Table  8.  Differences  among  the  treatment  condi- 
tions are  significant,  F = 5.34,  £<  .01.  Although  differences  among 
the  raters  are  also  significant,  the  interaction  between  the  two  factors 
is  not  significant  (F  = 1.93,  N.S.),  indicating  that  group  differences 
in  reported  strategies  are  not  a function  of  rater  bias.  Therefore, 
strategy  scores  were  averaged  across  raters  (the  inter-rater  reliability 
coefficient  is  .81).  The  resultant  group  means  are  shown  in  Table  9. 
Differences  between  the  training-plus-aids  and  the  training-only  groups 
are  significant  (t  (24)  = 2.93,  £ < .01),  and  the  differences  between  the 
training-plus-aids  and  the  control  group  are  also  significant  (t  (24)  = 
2.38,  £ < .05).  We  may  conclude,  then,  that  students  in  the  training-plus- 
aids  group  learned  significantly  more  of  the  self-direction  learning 
strategies  than  did  students  of  either  of  the  other  two  groups. 

Why  students  in  the  training-plus-aids  group  should  report  dif- 
ferent strategies  than  those  in  the  training-only  group  is  difficult  to 
understand.  One  explanation  that  seems  likely  relies  upon  the  fact  that 
* students  were  instructed  to  produce  summaries  "that  you  used  to  solve  the 

problem  you  just  worked  on."  As  we  have  already  seen  in  the  performance 
data,  students  in  the  training-only  group  performed  quite  poorly.  It 
seems  likely  that  they  were  not  making  use  of  the  self-directed  learning 
strategies  during  the  final  session.  The  fact  that  they  did  not  produce 
summaries  of  the  self-direction  strategies  does  not  necessarily  mean  that 
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Table  7 


Means  and  Standard  Deviations  of 
Strategy  Summary  Judgements 


Raters  Treatment  Condi tior 


Traininq  + Aids 

Trai ninq-Only 

Control 

I 

X 

1.5 

0.5 

0.6 

SD 

(1.3) 

( .7) 

( -9) 

II 

X 

3.5 

1.9 

2.9 

SD 

(1.8) 

(1.6) 

(1.4) 

III 

X 

3.0 

1.5 

1 .4 

SD 

(1.6) 

(1.4) 

(1.3) 
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Table  8 

ANOVA  Summary  Table  for  Strategy 
Summary  Judgements 


Source 

df 

MS 

F 

Between  Treatment  Condition 

2 

19.56 

5.34* 

Error^ 

36 

3.67 

Within  Raters 

2 

35.41 

37.55* 

Interaction 

4 

1.821 

1 .931 

Error, , 
w 

72 

.94 

£ < .01 

Table  9 

Averaged  Strategy  Summary 
Judgements 

Treatment  Condition 

Training  + Aids  Trai ni ng-Only  Control 

X 2.7  1 . 3a  1.6b 

SD  (1.4)  (1.0)  (0.9) 

d £<  .05  for  comparison  to  Training  + Aids  group. 
b £<  .05  for  comparison  to  Training  + Aids  group. 
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they  did  not  learn  the  strategies.  Rather,  they  may  have  simply  been 
obeying  the  instructions  by  not  describing  the  strategies  that  they  failed 
to  use. 


Discussion 

Our  analyses  of  the  data  on  self-direction  and  reading  effect- 
iveness found  no  significant  differences  due  to  group.  Therefore,  the 
results  did  not  support  either  of  our  research  hypotheses  (that  training 
in  self-directed  learning  would  improve  performance  and  that  use  of  the 
aids  system  would  further  improve  performance).  There  was,  however,  a 
non-significant  tendency  in  the  data  for  those  students  who  had  both 
received  training  and  had  the  use  of  the  aids  system  during  the  test  to 
perform  better  than  the  students  in  the  other  two  groups.  It  is  possible 
that  with  more  training  practice  or  with  more  subjects  these  results  might 
have  reached  significance.  It  is  noteworthy  that  the  training-plus-aids 
group  outperformed  the  other  two  groups  on  each  of  the  performance  measures 
we  took.  On  only  one  measure,  the  scores  for  summarized  strategies,  did 
the  performance  of  the  training-plus-aids  group  significantly  exceed  that 
of  the  other  two  groups.  Interpretati on  of  this  result  is  problematic. 
Those  students  who  were  both  trained  in  the  use  of  the  self-directed 
learning  aids  system  and  given  access  to  that  system  in  the  test  session 
were  able  to  later  summarize  the  principles  of  the  system  in  writing. 

Yet  other  students  who  had  received  the  same  training  (the  training-only 
group)  but  did  not  practice  with  it  in  the  last  session  were  no  better 
than  control  subjects  at  expressing  the  strategies. 

An  unexpected  aspect  of  our  results  was  the  (non-significant) 
difference  between  the  control  group  and  the  group  that  was  trained  in 
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the  use  of  the  aids  system  but  did  not  have  the  use  of  that  system  during 
the  test.  We  had  predicted  that  the  training-only  group  would  perform 
less  well  than  the  fraining-plus-aids  nroup,  but  better  than  the  control 
group.  Instead,  control  group  students  did  better  than  those  in  the 
training-only  group.  In  retrospect,  this  result  seems  quite  natural, 
students  in  the  training-only  group  received  training  in  the  use  of  one 
computer-based  aids  system  but  were  tested  on  their  facility  with  another. 

The  switch  in  systems  may  well  have  been  confusing,  and  this  could  have 
caused  their  performance  to  deteriorate.  Another  possibility,  however, 
is  that  training  in  the  use  of  the  experimental  aids  system  does  not 
improve  one's  ability  to  learn  in  a self-directed,  selective  way.  Perhaps, 
on  the  contrary,  it  creates  a kind  of  intellectual  dependence  on  the 
facilities  of  the  aids  system.  When  these  facilities  are  removed,  those 
who  have  been  trained  in  their  use  perform  less  well  than  those  who  have 
not  been  exposed  to  the  aids  system.  Given  the  nonsignificance  of  the 
results,  such  conclusions  are,  at  best,  merely  speculative. 

An  interesting  aspect  of  the  results  is  the  finding  that  different 
learning  measures  are  related.  The  two  measures  of  effective  learning 
(reading  time  and  errors)  are  significantly  correlated,  as  are  the  three 
measures  of  self-directed  learning  (two  selectivity  ratios  and  planning 
violations).  The  question  naturally  presents  itself:  Is  effective 
learning  related  to  self-directed  learning?  Of  six  possible  relation- 
ships, only  one  was  found  to  be  significant--selectivity  of  title  choice 
is  inversely  related  to  number  of  errors.  In  the  control  condition,  however, 
three  other  relationships  emerge.  Both  selectivity  ratios  are  inversely 
related  to  reading  times,  and  planning  violations  are  directly  related 
to  errors  made  in  attempting  the  task.  Why  should  these  relationships 
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appear  only  in  the  control  condition?  Scatter  diagrams  of  these  relation- 
ships show  that  the  control  condition  represents  the  expected  relationship. 
For  example.  Figure  2 plots  selectivity  of  title  choice  against  reading 
time.  For  students  in  the  control  condition,  the  less  selective  they  are, 
the  more  time  they  spend  reading.  This  relationship  does  not  seem  to 
hold  for  the  students  in  the  training-plus-aids  group,  however. 

The  results  of  the  research  reported  here  lend  support  to  two 
conclusions  that  we  believe  are  also  supported  by  other  recent  results 
in  cognitive  research.  The  first  is  that  human  learning  and  human 
thinking  are  not  very  general  processes,  but  are  always  closely  linked 
to  fairly  detailed  or  specific  situations.  The  second  conclusion  is  that 
human  learning  strategies  or  skills  are  highly  automatized;  as  a result, 
even  inefficient  learning  strategies  may  lead  to  superior  performance 
when  they  are  compared  to  the  use  of  non-automatic  strategies.  In  the 
remainder  of  this  paper  we  will  present  evidence  for  these  two  claims 
and  discuss  their  implications  for  training  research. 

A number  of  theorists  in  cognitive  science  (for  example,  Goldstein 
& Papert,  1977)  have  recently  proposed  that  models  of  human  thought  should 
include  few,  if  any,  general  processes.  Rather,  knowledge  and  thought  are 
best  represented  as  a collection  of  quite  specific  concepts,  concepts  that 
are  bound  to  particular  restricted  entities  in  particular  situations.  If 
this  claim  has  substance,  it  may  be  that  the  strategies  that  subjects  in 
our  training  groups  learned  were  somehow  closely  bound  up  in  their  minds 
with  the  topic  matter  of  the  example  learning  problems  that  were  used 
during  training.  In  the  training  sessions,  subjects  were  given  problems 
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on  the  Rules  of  the  Road.  The  test  session,  however,  used  a very  different 
(and  much  more  complicated)  problem  of  troubleshooting  a defective  device. 
Subjects  may  have  learned  the  strategies  that  they  had  been  taught  in  such 
a way  that  they  could  apply  them  only  to  Rules  of  the  Road  problems.  Some 
of  the  comments  made  by  the  subjects  in  their  written  evaluations  of  the 
experiment  seem  to  reflect  this  problem: 

"To  start  with  I made  up  4 goals  objectives  to  complete 
before  I could  complete  the  task.  Then  as  I worked  on  the 
information  I noticed  that  some  (most?)  of  my  objectives 
were  not  really  suited  to  the  subject." 

"This  problem  was  a little  different  in  regards  to 
strategy  than  others  at  1 point." 

"The  third  problem  set,  after  learning  goal  stack, 
objectives,  and  dependency  should  place  more  emphasis  on  the 
use  of  those  features.  The  two  problems  I had  in  this  phase 
of  the  training  were  much  too  easy  to  incorporate  all  features 
of  the  system." 

"If  the  practise  [sic]  problems  were  more  difficult  it 
would  help  repare  [sic]  the  learner." 

"More  practice  with  tougher  problems." 

"A  few  of  the  problems  (2nd  and  3rd  session)  were  pretty 
easy  and  I was  able  to  have  a few  common-sense  deductions 
about  the  answers.  I think  the  experiment  would  have  gone 
off  much  better  if  the  areas  studied  were  a little  more  dif- 
ficult and  challenging." 

If  this  process  (of  learning  strategies  only  with  respect  to  certain 
topic-matter  domains)  is  as  widespread  as  we  fear  it  is,  a number  of 
measures  should  be  taken  to  improve  the  results  in  attempting  to  teach 
such  strategies.  Two  methods  come  to  mind.  First,  the  applications  for 
which  the  strategies  are  intended  should  be  closely  examined.  If  they 
belong  to  a restricted  class,  then  the  practice  materials  for  training 
in  the  strategies  should  all  come  from  that  class.  Second,  if  the 
applications  are  not  members  of  a restricted  class,  then  as  wide  a 


-30- 


O 


variety  of  practice  problems  should  be  used  as  is  possible.  The  training 
process  will  almost  certainly  have  to  be  more  protracted  in  such  a case. 

The  second  general  conclusion  about  human  learning  that  we  have 
been  led  to  is  that  adults'  learning  strategies  have  been  highly  auto- 
matized. Rigney  (1978)  has  reviewed  evidence  on  this  issue.  If  this 
claim  is  true,  we  should  not  be  surprised  to  discover  that  our  experimental 
subjects,  who  should  have  been  using  superior  strategies,  did  not  do 
significantly  better  than  those  subjects  in  the  control  group.  First,  it 
is  possible  that,  for  many  of  our  experimental  subjects,  the  highly  over- 
learned old  inefficient  strategies  automatically  went  into  action  and 
competed  with  the  new,  less  well -learned  strategies  we  had  taught  them. 
Second,  if  some  experimental  subjects  were  able  to  use  the  new  strategies 
we  had  taught  them,  they  could  have  been  at  a disadvantage  with  respect 
to  the  control  subjects  who  used  their  old  strategies.  The  control 
subjects'  strategies  could  be  activated  automatically,  and  should  have 
required  little  conscious  control.  The  new,  unfamiliar  techniques  used 
by  the  experimental  subjects,  however,  would  surely  require  considerable 
conscious  control,  thus  reducing  the  processing  resources  available  for 
learning  and  problem  solution.  Viewed  in  this  light,  the  fact  that 
experimental  subjects  did  not  do  significantly  worse  than  control  subjects 
seems  to  support  the  essential  validity  of  the  learning  techniques  em- 
bodied in  the  aids  system. 
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APPENDIX 


Prescriptive  Schemata  Attained  as  a Result  of 
Training  in  Self-directed  Learning  Strategies 
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(!)  SELF-DI RECTED-LEARNING  (TASK) 
is  when 

BUILD-GOAL-STRUCTURE  (TASK) 

TASK-PURSUE  (TASK) 
end. 

(2)  BUILD-GOAL-STRUCTURE  (TASK) 

is  when 

ANALYZE  (TASK,  for  OBJECTIVES  (TASK))* 

PREREQUISITE-SEARCH  (For  EACH  (OBJECTIVE),  IN  OBJECTIVES) 
PREREQUISITE-SEARCH  (for  EACH  (OBJECTIVE),  in  CONTENTS) 
end. 

(3)  TASK-PURSUE  (TASK) 

is  when 

EXAMINE  (GOAL-STRUCTURE) 

UNTIL  (CHECKED  (EVERY  (OBJECTIVE)),  PURSUE  (OBJECTIVE)) 
TASK-ATTEMPT  (TASK) 
end. 

(4)  TASK-ATTEMPT  (TASK) 

is  when 

IF  (DO  (TASK),  then  QUITE,  else  SELF-DI RECTED-LEARNING  (TASK)) 
end. 


The  ANALYZE  sub-schema  has  not  yet  been  represented.  How 
people  are  able  to  discover  the  prerequisites  or  component  actions  of 
a task  is  not  well  understood. 
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(5)  PREREQUISITE-SEARCH  (for  GOALS,  in  SUBGOAL-SET) 

is  when 

FOR-EACH  (MEMBER,  of  SUBGOAL-SET ) , 

IF  (PREREQUISITE  (MEMBER,  for  GOAL), 

then  (SPECIFY-DEPENDENCY  (MEMBER,  to  OBJECTIVES-LIST) ) ) ) 
end . 

(6)  PURSUE  (GOAL)* 

is  when 

FOR-EACH  (SUBGOAL  (NECESSARY  (SUBGOAL,  to  GOAL))  in  GOAL-STRUCTURE, 
WHILE  (ANY  (UNSATISFIED  (SUBGOAL ' (NECESSARY  (SUBGOAL',  to 
SUBGOAL))))'  • 

PURSUE  (SUBGOAL')) 

TRIAL  (SUBGOAL  )) 
end. 

(7)  UNSATISFIED  (GOAL) 

is  when 

NOT  (CHECKED  (GOAL)) 

NOT  (ELIMINATED  (GOAL)) 
end. 

(8)  TRIAL  (GOAL) 

is  when 

ATTEMPT  (ACTION,  of  GOAL) 

EVALUATE  (GOAL) 
end. 


•This  structure  is  a variant  of  Rumelhart  & Ortony's  (1977) 
schema  for  TRYing,  a subschema  of  their  PROBLEM-SOLVING  schema. 
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(9)  EVALUATE  (GOAL) 


is  when 

IF  (NECESSARY  (GOAL,  to  HIGHER-GOAL), 

then  IF  (SATISFIED  (GOAL),  then  CHECK  (GOAL), 
else  TASK-PURSUE  (TASK), 
else  ELIMINATE  (GOAL,  from  GOAL-STRUCTURE)) 
end. 

(10)  ATTEMPT  (GOAL) 
is  when 

IF  (BELIEVE  (CAUSE  (ACTION,  SATISFIED  (GOAL))), 
then  DO  (ACTION), 

else  when  SUCCEED  (PREREQUISITE-SEARCH  (for  GOAL)), 

ATTEMPT  (PREREQUISITE  (GOAL))) 

end. 

According  to  the  first  of  these  schemata,  the  student  believes 
that  the  way  to  achieve  a task  through  self-directed  learning  is  first 
to  build  a goal  structure  and  second  to  pursue  the  task,  using  that  goal 
structure.  The  second  schema  listed  above  describes  what  is  involved 
in  building  a goal  structure.  One  analyzes  a task  for  objectives  (sub- 
goals necessary  for  the  performance  of  the  task),  then  one  searches 
for  prerequisite  relationships  among  these  objectives,  between  the 
available  information  resources  and  the  objectives,  and  among  the  rele- 
vant available  information  sources.  However,  the  schema  does  not  contain 
explicit  reference  to  the  process  of  adding  these  relationships  to  the 
goal  structure,  because  the  goal  structure  is  constructed  for  the  student 
by  the  program  that  aids  him  or  her  in  self-directed  learning.  The  fifth 
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schema  listed  above  is  an  essential  part  of  the  goal-structure-building 
schema,  since  it  specifies  how  the  search  for  prerequisites  is  conducted. 

The  second  major  part  of  self-directed  learning,  after  building 
a goal  structure,  according  to  the  above  schemata,  is  to  pursue  the  task. 
The  third  schema  above  gives  the  top-level  structure  for  task  pursuit. 

One  examines  the  newly  constructed  goal  structure  first;  then  one  pursues 
the  objectives  included  in  that  goal  structure  until  every  one  of  them 
has  been  checked.  (Checking  is  the  process  by  which  a student  marks  the 
attainment  of  a subgoal,  using  the  Aids  program  on  PLATO).  When  all  the 
necessary  objectives  have  been  checked,  the  student  attempts  the  task. 

If  the  attempt  fails  (see  schema  #4),  then  he  begins  the  self-directed 
learning  process  again,  reconstructing  or  modifying  the  goal  structure. 

The  pursuits  of  objectives  is  governed  by  the  sixth  schema  given 
above.  This  is  a recursive  procedure  that  traces  down  dependency  rela- 
tionships in  the  goal  structure.  When  a goal  is  found  that  has  no  pre- 
requisites, that  goal  is  subjected  to  a trial.  This  means  (see  #8,  9 
& 10)  that  the  student  does  an  action  to  bring  about  the  goal  and  then 
evaluates  the  results  of  that  action.  If  the  goal  is  satisfied,  he 
checks  the  goal  and  then  pops  back  to  the  appropriate  point  in  the 
procedure  that  is  pursuing  an  objective.  If  it  is  not  satisfied,  he 
looks  for  a new  way  to  pursue  his  overall  task.  If  the  attempt  reveals 
that  the  goal  was  unnecessary  to  the  attainment  of  its  higher  goal, 
then  it  is  dropped  from  the  goal  structure. 
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